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Abstract 

We construct a density estimator and an estimator of the distribution function in the 
uniform deconvolution model. The estimators are based on inversion formulas and kernel 
estimators of the density of the observations and its derivative. We derive point wise 
asymptotic normality, the point wise asymptotic biases, as well as an expansion of the 
mean integrated squared error of the density estimator. Some simulated examples are 
included. 
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1 Introduction 

Consider the general deconvolution model. Let Vi,..., V„ be i.i.d. observations, where V* = 
Yi + Zi and V and Zi are independent. Assume that the unobservable V have distribution 
function F and density /. Also assume that the unobservable random variables Zi have a 
known density k. Note that the density g of Vj is equal to the convolution of / and /c, so 
g = k * f where * denotes convolution. So we have 

/ OO 

k{x — u)f{u)du. (1.1) 

-OO 

The deconvolution problem is the problem of estimating f or F from the observations Vj. Later 
on we will restrict ourselves to uniform deconvolution where we require the distribution of the 
Zi to be uniform. 
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Several generally applicable methods have been proposed for this deconvolution model but 
let us review direct kernel density estimation first. Consider estimation of the density function 
g from the observations Xi, • • • , Xn- The kernel density estimator with kernel function w and 
bandwidth h > 0, is dehned by 


9nh{x) = - 

n 



( 1 . 2 ) 


For smooth g, essentially twice continuously differentiable, and symmetric w with integral one, 
we have 


^9nh{x) = 

= 9{x) - 
Vaignh{x) 


— OO 
1 


X — u 

h'°y— 


g{u)du 


h g (x) J u w{u)du + o{h ), 

1 1 
— g{x) / w^{u)du + o(—), 
nti ./ \nn/ 


as n —)■ cxD, h —)■ 0 and nh —)■ cx). For more on direct kernel density estimators see for instance 
Prakasa Rao (1983), Silverman (1986) and Wand and Jones (1995). 


The standard Fourier type kernel density estimator for deconvolution problems is based on 
the Fourier transform. For an introduction see for instance Wand and Jones (1995). Let w 
denote a kernel function and h > 0 a bandwidth. The estimator fnh{x) of the density / at the 
point X is dehned as 


fnh{x) 


Mt) 


1 

nh 


n 


i=i 


p - 
V h 


•> 


with 


where 


Vhiu) 


i_ Ms) 

2n (t>k{s/h) 


^-isu 


ds, 


n 

j=i 


and (fw and (fk denote the characteristic functions of w and k respectively. 

An important condition for these estimators to be properly dehned is that the characteristic 
function 0^ of the density k has no zeroes, which renders it useless for instance for uniform 
deconvolution. At the same time this shows that uniform deconvolution is a non standard 
deconvolution problem. In fact, Hu and Ridder (2004) argue that in economic applications 
the assumption of no zeros for (fk is not reasonable. If the error distribution has bounded 
support and is symmetric then its characteristic function will have zeros. They propose an 
approximation of the Fourier transform estimator in such cases. For other modihcations of the 
Fourier inversion method, applicable to uniform deconvolution, see Hall and Meister (2007) and 
Feuerverger, Kim and Sun (2008). 
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A second general approach is nonparametric maximum likelihood. The likelihood of F is 
equal to 

^ ^ poo 

n9W)=n/ k{Xi-t)dF(t). 

j=l j=ld-°o 

One can try to explicitly determine a distribution function F that maximizes this likelihood. 
This works for exponential deconvolution and for a special case of uniform deconvolution (if 
the distribution induced by F concentrated on [0,1]) where explicit expressions can be derived. 
In other cases a numerical maximization procedure is required. For recent results see the thesis 
of S. Donauer. 


A third general approach is provided by inversion. A selected group of deconvolution 
problems allows explicit inversion formulas of (ll.ip . expressing the density of interest / in 
terms of the density g of the data. In these cases we can estimate / by substituting for instance 
a direct density estimate of g, for instance the kernel estimate fll.2l) . in the inversion formula. 
In Van Es and Kok (1998) this strategy has been pursued for deconvolution problems where k 
equals the exponential density, the Laplace density, and their repeated convolutions. 

In the uniform deconvolution problem the error Z is Uniform[0,1) distributed. So in this 
particular deconvolution problem we assume to have i.i.d. observations from the density 

/ oo px 

I[o,i){x - u)f{u)du = f{u)du = F{x)-F{x-l). (1.3) 

■OO J x—1 

Groeneboom and Jongbloed (2002) consider density estimation in this problem. They propose a 
kernel density estimator based on the nonparametric maximum likelihood estimator (NPMLE) 
of the distribution function F and derive its asymptotic properties. Under the restriction 
that / is concentrated on the interval [0,1], and that / is bounded away from zero, its better 
performance compared to a more standard kernel estimator, discussed below, is noted. Our aim 
is to show that a kernel type estimator of / can be constructed that, for all /, not necessarily 
concentrated on [0,1], under some smoothness assumptions has the same asymptotic bias and 
variance as the one based on the NPMLE of F, cf. Theorem 12.31 and Remark 12.41 below. 

In this construction an inversion approach is employed. The inversion is based on fll.3p . 
In fact this will lead to two inversion formulas, yielding two possible estimators. We will then 
combine these estimators into an estimator with asymptotically minimal variance. We also 
construct an estimator for the distribution function F. The construction is very similar to 
that of the density estimator. For other estimators of the distribution function in uniform 
deconvolution we refer to Van Es (1991), Groeneboom and Wellner (1992) and Van Es and Van 
Zuijlen (1996). 


2 Uniform deconvolution 

2.1 Inversion formulas 

Inversion of the relation fll.3p is relatively simple. Surprisingly we get two different expressions 
which of course coincide for density functions g of the form fll.3p . 
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Lemma 2.1 If g is of the form then we have 


OO 


Fix) = ^g{x-j), 

j=0 

(2.4) 

OO 

Fix) = 1 -'^gix + j). 

(2.5) 


i = l 


Furthermore, assuming that f vanishes at plus and minus infinity, and that g is continuously 
differentiable, we have 


OO 

fix) = '^g'{x - j), (2.6) 

j=0 

OO 

fix) =-'^g'ix + j). (2.7) 

i=i 


Proof 

Note that formula fll.3p can be rewritten as F{x) = g{x) + F{x — 1) and, replacing x by 
a; + 1, as F{x) = —g{x + 1) + F(a; + 1). Iterating these formulas gives the hrst two inversion 
formulas for the distribution function F. Differentiating these formulas yields the two formulas 
for the density /. □ 

2.2 Estimation of the density function 

We construct our estimator using the two inversion formulas of Lemma 12.11 The fact that the 
two expressions for / and F in (12.Ih are equal, if g is of the form (II.3p . also follows from the 
fact that 

OO OO 

six + j)= ^^i^ + - ^ix + j -1)} = 1 . 

j=—oo j=—oo 

For an arbitrary density g, which is not of the form fll.3p . the inversions will in general not 
yield distribution functions or densities, nor will they coincide. In particular, if we substitute 
a kernel density estimator like fll.2p for g then we get different estimators of / from fl2.6p and 
02.71) . We get 

OO OO 

fnhix) = ^nhix - j) and /+^(x) = -J2 9nhix + ])■ (2.8) 

i=o j=i 

The hrst of these estimators has also been discussed by Groeneboom and Jongbloed (2002). 

We impose the following condition on the kernel function. 

Condition Wi 

The function w is a continuously differentiable symmetric probability density function with 
support [-1,1]- 
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Because of the bounded support of the kernel estimator gnh the sums in fl2.8l) are in fact hnite 
sums. Moreover, will be periodic with period one for x large enough and for x small 
enough. For instance, f~f^ is equal to the sum of g'^hiy) over the values y = x,x — l,x — 2,.... 
Once y is on the right hand side of the support of g'^f^ this sum does not change anymore if 
we replace x by a; + 1. Also note that vanishes for x smaller than the left endpoint of the 
support of g'^f^. Similarly vanishes for x larger than right endpoint of the support of 


Let us derive the kernel estimator. Groeneboom and Jongbloed (2002) show 
is asymptotically normally distributed. More precisely, as n —)■ cxo, h —)■ 0 and nh 
show 




that 

—>■ oo, they 


with 



However, by a similar proof it follows that 


- E/nh(a^)) ^ N{0,a^), 


with 

(72 = (1 — F{x)) J w'{uydu. 

Apparently the hrst estimator has a small variance for small values of x and the second estimator 
for large values of x. Hence it makes sense to combine the two. Consider 

= tfnhi^) + (1 - 

for some hxed 0 < f < 1. The following theorem establishes asymptotic normality and the 
asymptotic bias of this estimator. It also contains the results for the two estimators fl2.8l) above 
as special cases, taking t equal to zero and one. 

Theorem 2.2 Assume that Condition Wi is satisfied and that f is bounded on a neighborhood 
of X. Then, as n ^ oo, h ^ 0,nh ^ oo, 

- E/S(x)) 4 N{0,a^) 


with 


Furthermore, if f is twice continuously differentiable on a neighborhood of x then 

Ffnh{x) = fix) + ^h'^f'ix) [ v‘^wiv)dv + oihf). 
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Up to now t has been an arbitrary constant. It turns out that the expectation of the 
estimator does not depend on t. See fl4.24p in the proof Theorem fl2.2l) . However, we can 
minimize the asymptotic variance by choosing a specihc value for t. This variance is minimal if 
t equals 1 — F{x). The minimal value is F{x){l — F{x)) f w'(u)^du. Furthermore it turns out 
that if we substitute an estimator F„(x) of F, the pivotal estimator, in 1 — F{x) for t, that is 
consistent in mean squared error, then we will achieve the minimal variance. So we introduce 

fnhix) = (1 - Fn{x))f-,^{x) + Fn{x)f+i^{x). (2.9) 

The following theorem establishes asymptotic normality and the asymptotic bias of this esti¬ 
mator. A suitable estimator Fn{x) will be constructed in the next section. 

Theorem 2.3 Assume that Condition Wi is satisfied, that f is bounded on a neighborhood of 
X, and that Fn{x) is an estimator of F{x) with 

E{Fn{x)-F{x)y ^0. ( 2 . 10 ) 

Then, as n ^ oo, h ^ 0,nh ^ oo, we have 

'/^{fnh{x) - E fnh{x)) ^ N{0, cr^). 


with 

= F(x)(l - F{x)) j w'{ufdu. (2.11) 

Furthermore, if f is twice continuously differentiable on a neighborhood of x and 

E {Fn{x) - F(x)f = o(nh^) (2.12) 

then we have 

Ffnh{x) = f{x) + ^h'^f'ix) j v'^w{v)dv + o{h^). 

Remark 2.4 The theorem shows that the kernel density estimator fnh{x) has the same asymp¬ 
totic properties as the kernel smoothed NPMLE of F of Groeneboom and Jongbloed (2002) under 
the restriction that f is concentrated on the interval [0,1], and that f is bounded away from zero. 
However, in Section\^we show that the limit variance of the kernel smoothed NPMLE is in fact 
egual to li2.11\) . even if the restriction of the support of f to [0,1] does not hold. This means 
that the limit distibutions of the kernel smoothed NPMLE and our estimator coincide. Eor the 
estimators of Hall and Meister (2007) and Eeuerverger et al. (2008) the limit distributions are 
not known. 

Remark 2.5 Admittedly, the estimator / lg..9l) lacks the desirable properties that the estimates 
are nonnegative and that their integral is egual to one, which are guaranteed for the kernel 
smoothed NPMLE. By truncating at zero and rescaling to integral one these properties can be 
restored. Eor the effect of this normalization on the Li distance and other distances to the true 
density f, see Kaluszka (1998). 
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2.3 Estimation of the distribution function 


To combine the two density estimators in the previous section optimally we need an estimator 
Fn{x) of F{x). The construction of such an estimator is similar to the construction of the 
density estimator. 

The inversion formulas fl2.6p and fl2.7p can again be used to construct estimators 


(2.13) 


j=0 j=l 

By similar techniques as in the proof of Theorem 12.31 one can show 

- Ef-Ji)) 4 Af(0,Tf), 

with 


and 

with 


tI = F{x) / w{uYdu. 


- EF+(a:)) 4 iV(0,r4, 


42 = (1 — F{x)) / w{uf‘du. 


Now define the estimator (®) by 

The following theorem establishes asymptotic normality and the asymptotic bias of this esti¬ 
mator. 

Theorem 2.6 Assume that Condition Wi is satisfied. Then, as n ^ oo, h ^ 0,nh ^ oo, 

v4A(r«4) - Edh4) ” A'(o.y), 

with 

= (^CF{x) + (1 — t)^(l — F(x))^ j w{ufidu. 

Furthermore, if f is continuously differentiable on a neighborhood of x then 

EFnh{x) = F{x) + -h‘^f\x) [ v‘^w{v)dv -|- o(h^). 


The same steps, i.e. optimizing over t, that resulted in the density estimator (12.9p can be 
repeated to construct an improved estimator of F. Dehne Fnh{x) by 

Fnh{x) = (1 - Fn{x))FC^{x) + F„(x)F+^(x). (2.14) 

We get the following analogue of Theorem 12.31 Note that the rate of convergence is faster than 
in the density estimation case. 
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Theorem 2.7 Assume that Condition Wi is satisfied and that Fn{x) is an estimator of F{x) 
with 

E{Fnix)-F{x)f ^0. 

Then, as n ^ oo, h ^ 0,nh ^ oo, we have 

V^{Fnh{x) - EFnh{x)) 4 iV(0, r^), 

with 

= F{x){l — F{x)) j w{uf‘du. 

Furthermore, if f is continuously differentiable on a neighborhood of x and 

E {Fn{x) - F(x)f = o(nh^) (2.15) 


then we have 

EFnh{x) = F{x) + ^h‘^f{x) J v‘^w{v)dv + o{h‘^). 


Proof 

The fact that we can replace t = 1 — F{x) by a consistent estimator 1 — Fn{x) and the bias 
expansion also follow as in the corresponding parts of the proof of Theorem I2.31 □ 


As we will see in Section |3] the fnll snbtlety of this resnlt is not needed to combine the 
two density estimators in the way of the previons section. It tnrns ont that the plain average 
I snffices for that pnrpose if we consider point wise estimation. For global 

properties derived in Section ITTI we will see that t will have to depend on x. 


2.4 Mean integrated squared error of the density estimator 

Up to now we have considered point wise, i.e. for a hxed x, properties of the estimators. 
Assnming that / is sqnare integrable, an important measnre of the global performance of a 
density estimator is the mean integrated sqnared error, given by 

MISE„(h) = E jiUix) - f{x)fdx. (2.16) 

If we want to consider this global distance then we have to make snre that onr estimator fnh is 
sqnare integrable. If we nse a hxed weight t, independent of x, for combining ffi,^{x) an f^^^x), 
then this is certainly not trne becanse of the periodicity at pins or minns inhnity of ffii,{x) and 
fnhi^)- repair this as follows. If we nse the optimal trne weight t = 1 — F{x) the 

’’estimator” is sqnare integrable once F and 1 — F are sqnare integrable in the left and right 
tail respectively. This holds becanse the estimator ffy, has a hnite (random) left end point of 
its snpport. Similarly f^hi^) h^is a hnite right end point of its snpport. However, we still have 
to estimate this weight. As an estimator of the optimal weights we will nse an estimator 
with t dependent on x. In particnlar we will choose t = 1 — H{x) where if is a distribntion 
fnnction with sqnare integrable tails. So we will nse 

dfo’A) = (1 - H(x))F;;,(x) + 
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(2.17) 






By the same reasoning as above has square integrable tails, and thus so has the density 

estimator that uses this pivotal estimate of F for the weight. Of course there is no need to use 
the same bandwidth for the density estimators fnhi^) /^(a;) as for estimating the weights. 

The next theorem gives an expansion of fl2.16p which allows us to establish rate optimality 
of our estimator. 


Theorem 2.8 Assume that Condition Wi is satisfied, that f F(x)(l — F(x))dx < oo, that f 
is square integrable and twice continuously differentiable with bounded and square integrable 
second derivative. Furthermore, assume that Fn{x) is an estimator of F{x) with, as n ^ oo, 

[(E {Fn{x) - F{x))y^^dx = o{nh'^). (2.18) 


Then, as n ^ oo, h ^ 0 and nh ^ oo we have 


MISEn(h) = J j v'^w{v)dv^ H —^ 


F{x){l — F{x))dx / w (v) dv 


The next lemma ensures that we can use fl2.17l) as pivotal estimator. 


Lemma 2.9 Assume that f is differentiable, that f is bounded and continuous and that 
/{F(x)(l — F{x))Y/‘^dx < oo. Also assume that f f'{xfidx < oo and that f H(xfi(l — 
H(x)fidx < oo, i.e. H and 1 — H are square integrable in the left and right tail. Then, if 
n —)■ cx), h ^ 0 and nh ^ oo, we have, with equal to the pivotal estimator ^2.11^), 


(E- F(x)fflHx = 0(h‘) + 0( 


\nh 


(2.19) 


The lemma shows that if we use a bandwidth of the form h = cin~^F for the pivotal 
estimator fl2.17p then fl2.19p is of order _ Condition fl2.18p now requires the bandwidth h 
of the two density estimators to satisfy = o{nhfi). This is achieved by choosing h ^ ^ 

thus allowing the rate optimal bandwidth h = C 2 n~^F_ 


Remark 2.10 The lower bounds for the integrated squared error in deconvolution problems, 
as derived by Fan (1993), Theorem 2, also hold for uniform deconvolution. Feuerverger et al. 
(2009) use this observation to show that their density estimator is rate optimal over Sobolev 
classes of densities. If we compare our mean integrated squared error expansion with an optimal 
bandwidth of the form h = cn~^F fo lower bound for a Sobolev class corresponding to twice 
differentiable densities then we see that our estimator, for fixed f, also achieves the optimal 
rate ^ 
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3 A simulated example 

We use the average of and as pivotal estimator in (I2.9p and (I2.14p . Define 

Fn(^) = dh’ = i (•f’.rJ*) + Fi,(x)). (3.20) 

The asymptotic variance of Fn{x) is equal to | / w{u)‘^du/nh. Since the mean squared error 
equals the sum of the squared bias and the variance we have 

E(F„(i) - F(x) f = 0(h*) + 0(d), 

which asymptotically vanishes as long as h —)■ 0 and nh —)■ oo. If we choose a bandwidth of the 
form h = C\n~^F then the order of this mean squared error is minimized. The minimal order is 
^-4/5 _ Pqj, density estimators we choose a second bandwidth. We then have to ensure that 
(12.12P holds, i.e. we should ensure _ This means that the bandwidth h of fnh{x) 

should satisfy h S> This is not an essential restriction since the asymptotically optimal 

bandwidth that follows from Theorem 12.31 is of order is thus allowed. 

As an illustration we have simulated a sample of size n = 500 from the convolution of the 
standard normal density (/) and the uniform density. The kernel function used is the biweight 
kernel 

= ^(1 

The resulting estimates are given in Figures [T] and [2l For and we have used the 
bandwidth h = 1 and for F^ we have chosen h = 0.7. Indeed, we see that the original estimates 
are relatively accurate in one tail and almost periodic in the other tail. The combined estimate 
is accurate in both tails. 



Figure 1: The estimates and and the true density f,h = l. 


Next let us consider the estimator of the distribution function. If we use the specihc estima¬ 
tor Fn{x) given by fl3.20p then condition fl2.15p requires n~‘^F nh^, which means h 3> _ 

Again this is not an essential restriction since the asymptotically optimal bandwidth that follows 
from Theorem 12.71 is of order n~^F_ 

Figures [3] and m give the estimates and F„/j, based on the same sample of n = 500 

observations as above. Here the bandwidth use is h = 0.7. Again, the original estimates are 
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Figure 2: The estimate Fn, with h = 0.7, and the hnal estimate fnh- 




Figure 3: The estimates and and the true distribution function F. 

relatively accurate in one tail and almost periodic in the other tail. The combined estimate is 
accurate in both tails. Note the reduced variance in the tails of Fnh compared to that of Fn in 
Figure El 


4 Proofs 


4.1 Proof of Theorem 12.2 

Note that 




i=o 


w 


,/x-j-Xi 


i=l j=0 


h 


and 




nh"^ 


n oo 

1 ,/x+j-Xi 

w ' 




i=l j=l 


Write 


n ^ oo 


1=1 j=o 


,fx- j - Xi 


(i-oE 


w 


,/x + j - Xi 


i=i 


-'^Uih{x) 

n ^ 


i=l 


where 


Uih{x) = 


j=0 


,fx- j - Xi 
h 




w 


i=i 


,fx + j - Xi 
h 


(4.21) 
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Figure 4: The final estimate Fnh of F. 


First we compute the expectations of the estimators. By fl2.6p we have 




w 


,/x-j - Xi 


h 


^ oo 


w 


X — j — u 


g{u)du 



(4.22) 


and similarly 

^fnhi^) = JlJ (4.23) 

So the expectation of both f~j^{x) and f^hi^) is equal to the expectation of an ordinary kernel 
estimator based on direct observations from /. From fl4.22p and fl4.23l) we see that 

+ (1 - t)^fnhi^) = \ j W f (u) du. (4.24) 

The bias expansion in the theorem now follows by standard arguments. 

Similar to (I4.22p one can show EUih{x) = 0(1) if / is bounded on a neighborhood of x. 
The next lemma gives the even moments of Uih{x). 

Lemma 4.1 For m even we have for h —)■ 0 

^Uih{xr = j^{t^F{x) + i-ir{i-tr{i-F{x)) J w'{vrdv+o 




Proof 

Note that 


w 


,fx-ji - Xi 


h 


w 


X 


32 


Xi 


h 


0 


if Ji 7^ 32 i 3 i £ Zi, j 2 G Z and h < 1/2. Similarly it is readily seen that the products of terms 
w' { ) vanish if h < 1/2 and if the ji are not all equal. 
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Now write 




i=o 


,(x - j - Xj 
h 




w 


i=i 


,fx + j-Xi 
h 


h^' 




3=0 


i=i 


h 




oo « 

/ w\v)^g{x- j - hv)dv 

3=0 


CX3 « 

+ (-1)™(1 / w'(t;)”^^(a; + j - 

i=i '' 

+ (-1)”(1 - «)” / m'(!i)'“(l - f (x - /!!>))<;«) 

= + (-1)”(1 - «)”(! - F(X)) f w'(v)"'dv + o(^). 


□ 


For the variance of (^) by Lemma 14.11 


Var/^2(^) = - X&T{Uih{x)) = -(EUih{xf - {EUih{x)) 
{t^F{x) + (1 — t)^(l — F{x)) [ w'{vYdv. 


nh^ 


We will now check the Lyapnnov condition for - ~ ^Uih{x)) to be asymptoti¬ 

cally normal, i.e. for some 5 > 0 we have to check 


E\UM -EU,h{x)\^+^ 

n'5/2(Var(t/i,,(x)))i+'5/2 


^ 0 . 


Using (a+ 6)^ < 2^{a^ + h^) we get, for snitable constants ci and C 2 to be obtained from Lemma 

an 

E{U^h{x)-EU,h{x)Y ^ 2\EUrh{xf + {EU,h{x)Y) 8ci ^ ^ 

< -^^- - - ~ —^ u. 


?7,(Var(f/i/i(a;)))2 n{yax{Uih{x))Y nhc^ 

This proves asymptotic normality of (f^hix) — E f^l{x))/\J Var f^l(x) for hxed t. 


□ 
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4.2 Proof of Theorem 12.3 

We must show that we can replace t = 1 — F{x) by a consistent estimator. Write 
fnh{x) = (1 - Fn{x))f-^{x) + Fn{x)f+^{x) = + Rnh{x), 


where 


Now write 


where 


Rnhix) = {Fn{x) - F{x))Snh{x) and Snhix) = /+;,(a;) - 


2=1 


Snh{x) = -y^Wihjx), 

n < ^ 

1 

,(X- j -Xi 


j=-oo 


h 


The next lemma establishes some properties of Snh{x). 

Lemma 4.2 We have E Snh{x) = 0, EWih{x)^ = / w'{u)'^du and 


Vnh^Snhix) ^ n(o, 



(4.26) 

(4.27) 


The distributions of the random variables Wih{x) and Snh{.x) are independent of x. 


Proof 

The hrst statement follows from (I4.22p and (I4.23p . The second statement follows from a 
computation similar to the one in the proof of Lemma ITTl Asymptotic normality can be proved 
as in the proof of Theorem 12.21 

The fact that the distribution does not depend on x can be seen by writing 



x-j-Xi 

h 


OO 

)-5:>»'( 

j=-oo 


X - j - Yi 

h 



Given x and Yi this sum equals a periodic function with period one evaluated at Zj. Since Zj 
is Uniform[0,1) distributed its distribution does not depend on x and Yi. 

□ 


By fl2.10p we have Fn{x) — F{x) A 0 and hence by Slutsky’s theorem Vnh^Rnh{x) A 0. 
Furthermore by the Cauchy-Schwarz inequality 

EV^\Rnh{x)\ < V^{E{Fn{x) - F{x)f)F\E{Snh{x))Y^^ ^ 0 . 


This shows that \/nh^{fnh{x) — E fnh{x)) has the same asymptotic normal distribution as 
Vi^ifnh — E/^5j ^^^^\x)), which proves the hrst statement of the theorem. 
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To prove the second statement note that by fl4.24p and a standard argument in kernel 
estimation we have 

-^f{u)du = f{x) + 

Furthermore 


-h^f''{x) / v^w{v)dv + o{h^ 


(4.28) 


E/, 


(l-F(x)) 

nh 




w 


X — 

~h 


^\Rnh{x)\ < (E(F„(a;) -F(a;))2)V2(E(5„,(x))2)V2 

= <,(x4v)o(^)=o(ft^). 

Together (14.2811 and (14.2911 prove the second statement of the theorem. 


(4.29) 

□ 


4.3 Proof of Theorem 12.6 

We copy the proof of Theorem 12.31 Note that 


and 


Write 


where 


Fnhi.^) = ^9nh{x-j) 
j=0 


nh 


EE 

1=1 j=o 


w 


x-j-Xi 


Khi^) = 1 -J29nh{x+j) = 


i=i 


w 


i=l j=l 


h 


x + j-Xj 
h 


Fnh{x) = tFnhi.^) + (1 - i)Fnh{x) 


X- j - Xi 


n ^ cxD 


i=l j =0 


-^Vih{x) + l-t 

T) < ^ 


-(i-‘)E 

i=i 


„,iL±gzN,, + i_i 


n 


Z=1 


^ihix) = \ h 


x + j - Xi 


j=0 j=l 

First we compute the expectations of the estimators. By (12.6p we have 


2 OCJ 

j=0 




h 


j=0 


'X - J - u\ . , , 

w[ - - - jg[u)du 


OO ^ 

E l I /'X — u 
-r I W 


j=0 


h 


h 


^ 9 {u- i)du 


(4.30) 


= ^ / w(^^^']F{u)du 
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and similarly 




h 


F{u)du. 


(4.31) 


Since it is a convex combination of F~j^{x) and F^^{x) the expectation of F!fl{x) is also equal 
to fl4.30l) and fl4.31l) . 

The equivalent to Lemma [4.II for Vih{x) is 

E'4W’" = ^(*’”rW + (-l)'”(l-*)’”(l-f(i^))/«’(»')’”*'+ o()^) (4.32) 

which follows by replacing w' by w and replacing 1/h^ by 1/h in the proof. 

For the variance of (a:) we then get 

VarF«(a;) = ^ Var(fh;,(a;)) = - (Efhh(a^))') 

~ -^it^Fix) + (1 — t)^(l — Fix)) [ wivYdv. 
nh J 

The Lyapunov condition for “ EVi/i(a:)) to be asymptotically normal can 

be checked as in the proof of Theorem 12.31 This proves asymptotic normality of (F^2(^) ~ 

E(x))/ ^jYaxF^l{x) for hxed t. □ 


4.4 Proof of Theorem 12.8 

Recall that by fl4.26l) we have 


fnh{x) = fll ^^''^\x) + Rnh{x), 


where 

Rnhix) = {Fn{x) - F{x))Snh{x) and Snh{x) = fnhi^) “ fnhi^)' 

We decompose the mean integrated squared error as follows 

MISE„(h) = Je (^fil-^^^^\x) - fix) + Rnhix))" dx 

= y E (^fnh^^^'''^\x) - fix)) dx + j ERnhix)^dx (4.33) 

+ 2 y E [if^l-^^^'>\x) - fix))Rnhix))dx. (4.34) 

The mean integrated squared error of can be written as integrated squared bias plus 

integrated squared variance. We have 

y E ifil~^‘'''^\x) - fix))^dx = y iEff~^^^^\x) - fix))^dx + j Vai f^l~^^^^\x)dx. 
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We have already noted in (14 .24^ that the expectation of is equal to the expec¬ 

tation of a standard kernel estimator. By Theorem 2.1.7 of Prakasa Rao (1983), or the original 
proof of Nadaraya, we have the standard expansion for integrated squared bias of 

i.e. 

J - f{x)fdx = j f"{xfdx(^ j v‘^w{v)dv^ + o{h^). 

Next consider the integrated variance. We have, with Uih{x) as in f|4.21|) . 

^ Var(t/i;,(a;)) = ^{EUinixf - {EUiH{x)f), (4.35) 

where by the proof of Lemma [4.11 with t = 1 — F{x), 

EUihixr = ((1-^(2^))™ y w'{vrF{x-hv)dv+i-l)^F{xr J w'{vr{l-F{x-hv))dv'^ 


Now use 


to get 


F{x 


hv) = F{x) — hv 



thv)dt 


EUih{xY =^F{x){l-F{x)) jw'ivfdv 

— ^{{1 — F{x)y + F{x)‘^) [ f vw'{vYf{x— thv)dvdt. 
h J-i Jo 


The integral with respect to x of the hrst term is hnite by the condition E R < cx3. The integral 
with respect to x of the second term is hnite by the fact that |(1 — F(x))^ -|- F{xY\ is bounded 
by two and Fubini’s theorem. Similarly, for the term EUih{x) in fl4.35p we get by the Cauchy 
Schwartz inequality and Fubini’s theorem 


{EUih{x)Ydx = 


"l F n2 

/ vw'{v)f{x — thv)dvdt\ dx 


< 


J-1 JO 
1 ^1 


»i 


v‘^w' {yYdvdt 
v^w'{vYdvdt 


'-wo 
>1 /•! 



J -1 Jo 

= 2 y v^w'{vYdv j f{x)‘^dx. 


/-I JO 

7‘ 

-1 Jo 


f{x — thvYdvdtjdx 
f{x — thvYdxdvdt 


Finally this gives 

y Vai f^]^^^''^\x)dx = ^F{x){l - F{x)) J wjvfdv + o(^-^y 
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For the integrated expected squared remainder term in fl4.33p we have 


ERnhixfdx = y E {Fn{x) - F{x)ySnh{xydx 

< j{E {Fn{x) - F{x)yy/\ESnh{x)Y^dx 
1 


since by Lemma [4.21 




, 4 ^ 1/2 _ 






nE WihixY + 3n(n - 1)(E hFi;,(a;))' 


1/2 


L 


n(^ / w'{v)'^dvj + 3n{n — / w'{vfdv 


1/2 


= o 


nh^J 


The proof of the theorem is completed by noting that the cross product term fl4.34p is negligible 
with respect to the hrst term fl4.33p by the Cauchy Schwartz inequality. □ 


4.5 Proof of Lemma 12.9 

Write 




’if"'/!) - F(x) = F/r’/i) - - F(x) 


By the triangle inequality we have 


(E(Fir’T)-F(x))‘) < (E(F//'"'»(x)-EFi/'“'»(a:))-) + (ef<,^J’‘»(x) - F{x) 

So by (a + 6)^ < 2(a^ + b‘^),a,b > 0, we also have 

(e (f ‘f (x) - f ( 1 ))“) ‘''T 2 (e (i) - E f ‘f (I))-) ‘''T 2 (e F‘f <'» (a:) - F(a:)) 

Hence it suffices to prove the bound of the lemma for the fourth power of the error and the 
usual square of the bias separately. 

In the proof of Theorem 12.61 we have seen that (x) can be written as 






= -'^Vih{x) + H{x) 
n 


i=l 


where 


j=0 


w 


— H{x) w 
i=i 


X +j - X, 


and that we have 


EF 






X — u 


F{u)du. 
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Following the same arguments as in the proof of Theorem 2.1.7, the MISE expansion for kernel 
estimators, of Prakasa Rao (1983) we have 

/ oo -1 poo poo 2 

{EF^f^'^^\x) — F{x)Ydx = -h‘^( f'{xYdxj( v‘^w{v)dvj +o(h^). (4.36) 

-OO ^ ^ j —OO ' ^ J —oo '' 

In order to deal with the error part we wite 

i=l 

where Vih(x) = Vih{x) — EVih{x). Since EVih{x) equals zero we have 

E (^E '4(^))‘ = 4;E (r.-.(^))' + 6 ^(e (n.(x))y. 

i=l 

From fl4.32p we get 

4e ({■■',,.( 1 ))'' ~ ^E (r„.(i))'‘ ~ ((1 - H(x))^F(x) + H(xni - F(x))) 

and 

y=3^ 

for certain constants Ci and C 2 - Since the square roots of the functions on the right hand side 
are integrable we now have 

jT (e(F™“'»(i) - EFir"(i))y'hi = o(T) 

Summarizing we get 

/ ^ / \ 1/2 / 1 \ 

(e(F™‘'»(i) - F(x))*) dx = 0(h^) + 0(^—), 

which completes the proof. □ 


y^(Var(V-ift(x)))' ~ ^ ((l-ff(a))=F(x)+J?(l)=(l-F(a)))t 



5 The limit variance of the smoothed NPMLE 

In this section we will show that he limit variance of the smoothed NPMLE is equal to the 
limit variance of our optimally combined kernel estimator. 

Let the distribution induced by F have support [0, M) for some M > 0 and let m denote the 
largest integer strictly smaller than M + 1. The asymptotic variance of the smoothed NPMLE 
in Theorem 2 in Groeneboom and Jongbloed (2003) is, in our notation, dehned as 

= I™ [ (5.37) 

hiO J 
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with the function Oh^t^F-, for 0 < t < M, dehned by 


n / E^o(l-^(^ + *)X(^-(^ + *)) , if a: e [0,1], fc = 0, 

^ I -EtJ<(^-(^ + *)) + W(^) , ifa:e[0,l], fc = l,...,m, 

where tCh(-) = w{-/h)/h. 

Lemma 5.1 The asymptotic variance ([5.371 ) is equal to F{x){l — F{x)) f w'(u)^du. 
Proof We write the integral in fl5.37l) as 

/ m 

^hxpdG = XI / ^ItA^ + + k)dx 

k=0 


= / (F(a;) — F(a: — 1)) {1 — F{x))w'f^{t — x) + {1 — F{x + l))w'f^{t — x — 1) 
Jo f 

+ (1 — F{x + 2))w'i^{t — a: — 2) + ... + (1 — F{x + m))w'i^{t — x — m)] "^dx 

+ f {F{x + 1) — F{x)) — F{x)w'i^{t — x) + {1 — F{x + l))w'f^(t — X — 1) 

Jo f 

+ (1 — F(a; + 2))w'i^{t — x — 2) + ... + {1 — F{x + m))w'i^{t — x — m)] "^dx 


+ / (F(x + 2) — F(a; + 1)) — F{x)w'i^{t — x) — F{x + l)w'f^{t — x — 1) 

Jo f 

+ (1 — F{x + 2))w'i^{t — X — 2) + ... + (1 — F{x + m))w'i^{t — x — m)] "^dx 


+ / (F(x + m) — F(x + m — 1)) — F{x)w'y^{t — x) — F{x + l)w'j^{t — x — 1) 


F{x + 2)w'f^{t — X — 2) + ... — F(x + m)w'f^{t — x — m) 


dx. 


The next step is to write out the squares, which we leave to the reader. Let / < t < / + 1 for 
some integer Z, then, since 0 < f — Z < 1 and x G [0,1], only the terms containing w'^{t — x — Z)^ 
will yield a non zero contribution for h small enough. This contribution is, for h to zero, equal 
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to 


,1 I 


^(F(x + j) — F{x + j — 1))(1 — F{x + — X — lydx 


'0 j=0 


r*l m 


+ / {F{x + j) — F{x + j — l))F{x + — X — l)‘^dx 


° j=i+i 


= / F{x + 1){1 — F{x + — X — l)'^dx 

Jo 

+ f (1 — F{x + l))F{x + lYw'y^{t — X — l^dx 
Jo 

F(t)(l - F{t)f + (1 - F{t))F{tf^ j w\ufdu 


1 


= Y^Fm-m) w'iurdu. 


Here we have used an expansion of the integral which is standard in kernel estimation theory. 
Taking the limit for h to zero as in fl5.37p now yields the result. □ 
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